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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS. 
WHICHEVER IS LONGER. FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If NO period for reply Is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 Responsive to communication(s) filed on 21 September 2007 , 
2a)S This action is FINAL. 2b)n This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11. 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim{s) 1,11-22.24.34-45 and 47-52 is/are pending in the application. 

4a) Of the above claim (s) 13-21 and 36-44 is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) 13 Claim(s) 1.11.12,22.24.34.45 and 47-52 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) n The specification is objected to by the Examiner. 

10)^ The drawing{s) filed on 21 September 2007 is/are: a)El accepted or b)n objected to by the Examiner. 
Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a), 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 !)□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * c)\J None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.n Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/IVIail Date. . 

3) □ Infomiation Disclosure Statement(s) (PTO/SB/08) 5) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 
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DETAILED ACTION 

This is a Final Office Action on the merits. This action is responsive to 
Amendments/Remarks, which was filed on 09/21/2007. 

Claims 1 . 1 1-22. 24, 34-35, 45 and 47-52 are pending. Claims 2-10, 23, 
25-33, and 46 are previously cancelled. Claims 13-21, and 36-44 are withdrawn 
from consideration (Non-elected claims). Claims 1 1 , 34, and 47-52 are previously 
amended, with claims 1,12, 22, 24, 35, and 45 being the independent claims. 
Applicant has amended independent claims 1, 12, 22, 24, 35, and 45. 

Effective filing date is 11/03/2003, (Assignee: Adobe). 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained thougt) the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

Claims 1, 11, 12, 22, 24, 34-35, 45, and 47-52 are rejected under 35 
U.S.C. 103(a) as being unpatentable Coifman et al. US 20040254791 A1, 
Provisional No. 60/451,024 filed 03/01/2003 (hereinafter Coifman), in view of 
Reintles e t al. US 20020067854A1 filed 12/01/2000 (hereinafter Reintjes). 
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Regarding independent claim 1, Coifman teaches: (as amended) 

A computer-implemented method for generating an audio- 
based represented electronically as a digital audio file, the audio- 
based form including one or more data fields, 

(See Coifman at Fig. 2-3 and Para 29-30, and Para 39, discloses a speech 
recognition system, includes audio-based form including one or more data fields 
and form database. 

defining structural information including a name for each of the one 
or more data fields and a description of a type of user data expected 
to be provided for each of the one or more data fields; 
(See Coifman at Fig. 3 and Para 41, discloses a speech recognition system, 
includes audio-based form including one or more data fields and forni database; 
whereby input fields that are related to other input fields within the overall 
electronic form. As illustrated in Fig. 3, the context specific sub-databases 381 
(pertaining to the medical findings field) and 382 (pertaining to the medical 
interpretations field) may include contextually intertwined text strings that the 
speech recognition system of the present invention must identify and properly 
select so as to achieve the efficiencies of the present Invention. This allows user 
to include a name and a description of a type of user data expected for each of 
the data fields.) 

where the audio-based form comprises audio signals recording 
a voice speaking a name of a data field followed by a pause during 
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which a user can speak the user data expected to be provided for 
the data filed. 

(See Coifman at Para, discloses input speech is provided to the speech 
recognition systenn via a voice collection device; 

Also see Coifman at Fig. 3 and Para 41, discloses a speech recognition 
system, includes audio-based form including one or more data fields and form 
database; whereby input fields that are related to other input fields within the 
overall electronic fomi; 

Also see Coifman at Fog. 3 and Para 32, discloses text input may need to 
be input into a medical form 310, that includes a patient's name, shown In 
computerized form field 315, the patient's address, shown in computerized form 
field 318, the patient's phone number, shown in computerized form field 320, and 
the patients age, shown in computerized form field 320. Sub-databases 371 , 372 
and 373 shown In FIG. 3 are specific examples of the general field sub- 
databases 271, 272 and 273 of FIG. 2. These sub-databases provide first-pass 
text strings for matching speech input provided by the doctor when populafing 
form fields 315, 318 and 328 (FIG. 3) respectively. 

the audio-based form, and encoding the zoning and structural 
information in one or more audio signals; and incorporating the one 
or more audio signals including the encoded zoning and structural 
information into the audio-based form. 
(See Coifman at Para, discloses input speech is provided to the speech 
recognifion system via a voice collecfion device; 
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Also see Coifman at Fig. 3 and Para 41, discloses a speech recognition 
system, includes audio-based form including one or more data fields and form 
database; whereby input fields that are related to other input fields within the 
overall electronic form; 

Also see Coifman at Fog. 3 and Para 32, discloses text input may need to 
be input into a medical form 310, that includes a patient's name, shown in 
computerized form field 315, the patient's address, shown in computerized form 
field 318, the patient's phone number, shown in computerized form field 320, and 
the patients age, shown in computerized form field 320. Sub-databases 371, 372 
and 373 shown in FIG. 3 are specific examples of the general field sub- 
databases 271 , 272 and 273 of FIG. 2. These sub-databases provide first-pass 
text strings for matching speech input provided by the doctor when populating 
form fields 315, 318 and 328 (FIG. 3) respectively. 

To support the above interpretation, the Examiner reads the claimed 
consistence with Applicant's Specifications, which states, ''zoning and structural 
information corresponding to tlie form sliown in FIG. 1. Tlie form's author defined 
a name, location and data type for each field. For example, the Employee Name 
field" at Page 8 Lines 13-15. . 

In addition Coifman does not expressly teach, but Reinties teaches: 

the method comprising: defining zoning information 
identifying a temporal location and temporal dimensions of the one 
or more data fields. 
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(See Reintjes at Para 34-42, discloses the zoning defining by temporal rule 
selection. 

Also see Reintjes at Para 32-34, discloses fill out forms from top to bottom 
and from page 1 to page N in order. For example, address and name fields 
contain collections of short strokes with certain characteristics (printed letters) 
and numeric fields (SS#, zip codes etc.) that can be identified as such in many 
cases. In this case, the content of the data, as well as its location, can be used to 
make decisions regarding the form. If there are check boxes or circled entries 
(e.g. Sex M/F), they can be used as reliable indicators of which page is being 
used as well as the forms paper alignment on the clipboard. For the purposes of 
this discussion, data that can be identified based on the foregoing characteristics 
will the referred to as "content identifiable data." To support the above 
interpretation, the Examiner reads the claimed consistence with Applicant's 
Specifications, which states, "zon/ng and structural information corresponding to 
tlie form sfiown in FIG. 1. The form's author defined a name, location and data 
type for each field. For example, the Employee Name field" at Page 8 
Lines 13-15. ) • 

It would have been obvious to a person of ordinary skill in the art at the 
time the invention was made to have modified Mukherjee's zoning information 
identifying a temporal location and temporal dimensions of the one or more data 
fields, to include a means of utilizing a speech recognition system, includes 
audio-based form including one or more data fields and form database of 
Coifman. One of the ordinary skills in the art would have been motivated to 
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modify this combination, because both Coifman and Reintjes teach method of 
electronic form filling, it would have been obvious to one skilled in the art to 
substitute one method for another (i.e. pen based input for audio input) to 
achieve the predictable result of accurately transcribe the speech into to text (see 
Coifman at Para 20). Also, It would be desirable to have an electronic form is 
being filled out by an individual, that is automatically identifying the form and 
individual pages so that data input by an individual could be automatically 
associated with the correct page of the correct form (see Reintjes at Para 6). 

Claim 11, Coifman teaches: 

wherein data entered on the form by a user can be extracted 
from the audio-based form based on the encoded zoning and 
structural information without access to a source of zoning or (See 

(Coifman at Para, discloses input speech is provided to the speech recognition 

system via a voice collection device; 

Also see Coifman at Fig. 3 and Para 41, discloses a speech recognition 

system, includes audio-based form including one or more data fields and form 

database; whereby input fields that are related to other input fields within the 

overall electronic form.) 
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Regarding independent claim 12, (as amended) 

the rejection of claim 1, which cites above is fully incorporated and 
is similarly rejected along the same rationale. 
In addition, Coifman teaches: 

generating a form definition defining the audio-based form, 
wherein audio data entered into the audio-based form by a user can 
be extracted from the audio-based form based on the encoded 
zoning and structural information without access to a source of 
zoning or structural information external to the audio-based form. 

(See Coifman at Fig. 3 and Para 41, discloses a speech recognition system, 

includes audio-based form including one or more data fields and form database; 

whereby input fields that are related to other input fields within the overall 

electronic form; 

Also see Coifman at Fog. 3 and Para 32, discloses text input may need to 
be input into a medical form 310, that includes a patient's name, shown in 
computerized form field 315, the patient's address, shown in computerized form 
field 318, the patient's phone number, shown in computerized form field 320, and 
the patients age, shown in computerized fomri field 320. Sub-databases 371 , 372 
and 373 shown in FIG. 3 are specific examples of the general field sub- 
databases 271 , 272 and 273 of FIG. 2. These sub-databases provide first-pass 
text strings for matching speech input provided by the doctor when populating 
form fields 315, 318 and 328 (FIG. 3) respectively. 
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Regarding independent claim 22, (as amended) 

the rejection of claim 12, which cites above, and is fully incorporated 
and is similarly rejected along the same rationale. 

Regarding independent claim 24, (as amended) 

is directed toward a computer program product perfomiing the 
method of claim 12, which cites above and is similarly rejected under the 
same rationale. 

Claim 34, Coifman teaches: 

wherein data entered on the audio-based form by a user can 
be extracted from the audio-based form based on the encoded 
zoning and structural information without access to a source of 
zoning or structural information external to the audio-based form. 

(See Coifman at Para, discloses input speech is provided to the speech 

recognition system via a voice collection device; 

Also see Coifman at Fig. 3 and Para 41, discloses a speech recognition 

system, includes audio-based form including one or more data fields and fomi 

database; whereby input fields that are related to other input fields within the 

overall electronic form; 

Also see Coifman at Fog. 3 and Para 32, discloses text input may need to 

be input into a medical form 310, that includes a patient's name, shown in 

computerized fonn field 315, the patient's address, shown in computerized fomi 
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field 318, the patient's phone number, shown in computerized form field 320, and 
the patients age, shown in computerized fomri field 320. Sub-databases 371 , 372 
and 373 shown in FIG. 3 are specific examples of the general field sub- 
databases 271, 272 and 273 of FIG. 2. These sub-databases provide first-pass 
text strings for matching speech input provided by the doctor when populating 
form fields 315, 318 and 328 (FIG. 3) respectively. 

Regarding independent claims 35 and 45, (as amended) 

are directed toward a computer program product performing the 
method of claim 12, which cites above and is similarly rejected under the 
same rationale. 

Claim 47, Coifman teaches: 

encoding instructions indicating where and how to transmit 
user data extracted from the audio-based form into one or more 
audio signals; and incorporating the one or more audio signals 
including the encoded instructions into the audio-based form. 

(See Coifman at Para, discloses input speech is provided to the speech 

recognition system via a voice collection device; 

Also see Coifman at Fig. 3 and Para 41, discloses a speech recognition 

system, includes audio-based form including one or more data fields and fomn 

database; whereby input fields that are related to other input fields within the 

overall electronic form; 
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Also see Coifman at Fog. 3 and Para 32, discloses text input may need 
to be input into a medical form 310, that includes a patient's name, shown in 
computerized form field 315, the patient's address, shown in computerized form 
field 318, the patient's phone number, shown in computerized form field 320, 
and the patients age, shown in computerized fomri field 320. Sub-databases 
371 , 372 and 373 shown in FIG. 3 are specific examples of the general field 
sub-databases 271, 272 and 273 of FIG. 2. These sub-databases provide first- 
pass text strings for matching speech input provided by the doctor when 
populating form fields 315, 318 and 328 (FIG. 3) respectively. 

Claim 48, Coifman teaches: 

encoding instructions indicating where and liow to transmit 
user data extracted from the audio-based form into one or more 
audio signals; and incorporating the one or more audio signals 
including the encoded instructions into the audio-based form. 

(See Coifman at Para, discloses input speech is provided to the speech 

recognition system via a voice collection device; 

Also see Coifman at Fig. 3 and Para 41, discloses a speech recognition 

system, includes audio-based form including one or more data fields and fomri 

database; whereby input fields that are related to other input fields within the 

overall electronic fomi; 

Also see Coifman at Fog. 3 and Para 32, discloses text input may need to be 
input into a medical fomri 310, that includes a patient's name, shown in 
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computerized form field 315, tlie patient's address, shown in computerized form 
field 318, the patient's phone number, shown in computerized form field 320, and 
the patients age, shown In computerized fonn field 320. Sub-databases 371 , 372 
and 373 shown in FIG. 3 are specific examples of the general field sub- 
databases 271, 272 and 273 of FIG. 2. These sub-databases provide first-pass 
text strings for matching speech input provided by the doctor when populating 
form fields 315, 318 and 328 (FIG. 3) respectively. 

Claim 49, Coifman teaches: 

encoding instructions indicating where and how to transmit 
user data extracted from the audio-based form into one or more 
audio signals; and incorporating the one or more audio signals 
including the encoded instructions into the audio-based form. 
(See Coifman at Para, discloses input speech is provided to the speech 
recognition system via a voice collection device; 

Also see Coifman at Fig. 3 and Para 41 , discloses a speech recognition 
system, includes audio-based form including one or more data fields and forni 
database; whereby input fields that are related to other input fields within the 
overall electronic form; 

Also see Coifman at Fog. 3 and Para 32, discloses text input may need 
to be input Into a medical form 310, that includes a patient's name, shown in 
computerized form field 315, the patient's address, shown in computerized form 
field 318, the patient's phone number, shown in computerized forni field 320, 
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and the patients age, shown in computerized form field 320. Sub-databases 
371 , 372 and 373 shown in FIG. 3 are specific examples of the general field 
sub-databases 271, 272 and 273 of FIG. 2. These sub-databases provide first- 
pass text strings for matching speech input provided by the doctor when 
populating form fields 315, 318 and 328 (FIG. 3) respectively. 

Claim 50, Coifman teaches: 

encoding instructions indicating where and how to transmit 
user data extracted from the audio-based form into one or more 
audio signals; and incorporating the one or more audio signals 
including the encoded instructions into the audio-based form. 
(See Coifman at Para, discloses input speech is provided to the speech 
recognition system via a voice collection device; 

Also see Coifman at Fig. 3 and Para 41, discloses a speech recognition 
system, includes audio-based form including one or more data fields and form 
database; whereby input fields that are related to other input fields within the 
overall electronic form; 

Also see Coifman at Fog. 3 and Para 32, discloses text input may need 
to be input into a medical form 310, that includes a patient's name, shown in 
computerized form field 315, the patient's address, shown in computerized form 
field 318, the patient's phone number, shown in computerized form field 320, 
and the patients age, shown in computerized form field 320. Sub-databases 
371, 372 and 373 shown in FIG. 3 are specific examples of the general field 
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sub-databases 271, 272 and 273 of FIG. 2. These sub-databases provide first- 
pass text strings for matching speech input provided by the doctor when 
populating form fields 315, 318 and 328 (FIG. 3) respectively. 

Claim 51, Coifman teaches: 

encoding instructions indicating where and how to transmit 
user data extracted from the audio-based form into one or more 
audio signals; and incorporating the one or more audio signals 
including the encoded instructions into the audio-based form. 
(See Coifman at Para, discloses input speech is provided to the speech 
recognition system via a voice collection device; 

Also see Coifman at Fig. 3 and Para 41 , discloses a speech recognition 
system, includes audio-based form including one or more data fields and form 
database; whereby input fields that are related to other input fields within the 
overall electronic form; 

Also see Coifman at Fog. 3 and Para 32, discloses text input may need 
to be input into a medical form 310, that includes a patient's name, shown in 
computerized fomri field 315, the patient's address, shown in computerized form 
field 318, the patient's phone number, shown in computerized form field 320, 
and the patients age, shown in computerized form field 320. Sub-databases 
371, 372 and 373 shown in FIG. 3 are specific examples of the general field 
sub-databases 271, 272 and 273 of FIG. 2. These sub-databases provide first- 
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pass text strings for matcfiing speech input provided by the doctor when 
populating fonn fields 315, 318 and 328 (FIG. 3) respectively. 



Claim 52, Coifman teaches: 

encoding instructions indicating where and how to transmit 
user data extracted from the audio-based form into one or more 
audio signals; and incorporating the one or more audio signals 
including the encoded instructions into the audio-based form. 
(See Coifman at Para, discloses input speech is provided to the speech 
recognition system via a voice collection device; 

Also see Coifman at Fig. 3 and Para 41, discloses a speech recognition 
system, includes audio-based form including one or more data fields and form 
database; whereby input fields that are related to other input fields within the 
overall electronic form; 

Also see Coifman at Fog. 3 and Para 32, discloses text input may need 
to be input into a medical fomn 310, that includes a patient's name, shown in 
computerized fomn field 315, the patient's address, shown in computerized form 
field 318, the patient's phone number, shown in computerized form field 320, 
and the patients age, shown in computerized form field 320. Sub-databases 
371 , 372 and 373 shown in FIG. 3 are specific examples of the general field 
sub-databases 271, 272 and 273 of FIG. 2. These sub-databases provide first- 
pass text strings for matching speech input provided by the doctor when 
populating form fields 315, 318 and 328 (FIG. 3) respectively. 
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It is noted that any citations to specific, pages, columns, lines, or figures in 
the prior art references and any interpretation of the references should not be 
considered to be limiting in any way. A reference is relevant for all it contains 
and may be relied upon for all that it would have reasonably suggested to one 
having ordinary skill in the art. See, MPEP 2123. 

Response to Arguments 
The Arguments filed on 09/21/2007 has been fully considered but they are 
not persuasive. Applicant's amendment necessitated the new ground(s) of 
rejection presented in this Office action . 

Conclusion 

Accordingly. THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this aiDtion. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
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the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Quoc A. Iran whose telephone number is 
571-272-8664. The examiner can nomially be reached on Monday through 
Friday from 9 AM to 5 PM EST. If attempts to reach the examiner by telephone 
are unsuccessful, the examiner's supervisor, Doug Hutton can be reached on 
571-272-4137. The fax phone number for the organization where this application 
or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). 

/Quoc A, Tran/ 
Patent Examiner 
Art Unit 2176 
12/04/2007 

l(Dou0 HuttonJ 
Doug Hutton 
Supervisory Primary Examiner 
Technology Center 2100 



